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METHOD AND SYSTEM FOR RECOGNIZING TIME 

BACKGROUND 

[001] Handwriting recognition software has made it possible to digitally capture 

5 handwriting and transform it into digital characters using an input capture device and a 
computer. The capture device may be a flat panel device that allows a user to enter 
normal handwritten notes onto a piece of paper attached to the capture device while 
information about the coordinates of the pen strokes is digitally recorded by the capture 
device. The capture device can later upload the digitally recorded handwritten notes 
10 into a computer where an uploading program receives and stores the handwriting 
strokes in memory, resulting in two copies of a document, namely the original 
handwritten version and a second, digitally encoded version. 

[002] Digital handwriting capture is useful when data must be entered into a 

15. computer program for processing, but original handwritten copies must be retained for 
legal or verification purposes. In these instances, it would be helpful to have 
handwriting automatically transformed into digital characters and transferred to a 
computer program without manual data entry. This may be achieved by placing a 
printed paper form with clearly defined input fields on a capture device, digitally 
20 capturing the handwritten notes in these input fields on the capture device, and 

uploading the digital notes to the computer. A recognition program may then interpret 
the digitally recorded handwritten notes within these input fields and transform them 
into a digitally encoded representation, which can be automatically entered into the 
computer program in the same manner as if the notes were manually entered via a 
25 keyboard. 

[003] However, the problem with these systems is that handwriting recognition 

software is generally not 100% accurate. Conventionally, an accuracy of approximately 
90% can be reached, but only after a user has gone through a series of arduous user- 
30 specific training sessions that allow the recognition software to adjust to the style of a 
particular user. So, time is lost by having to train the software. But, even so, there may 



1 of 18 



11884/404501 

still be multiple transformation errors because of the margin for error in current 
handwriting recognition software. 

[004] Errors are particularly prevalent when a user enters time information in 

5 numerical form. For example, if an input field is provided for time entry and no format 
is given, a user may enter the time in one of many formats. For example, midnight may 
appear as 0:00, 12:00, 12:00 AM, or 12.00. The ambiguous nature of the time field 
results in difficulty in resolving handwritten input as a time understood by the computer 
program that will later use this information. Additionally, even if separate fields are 
10 provided for hours and minutes and the expected format is clearly marked on the 

printed form, users may still unintentionally enter the time in an incorrect format if they 
ignore this additional instruction. 

[005] Finally, even if users correctly follow the format required, other difficulties 

15 . may arise when resolving individual numbers. Different cultures may write numbers 
differently. For example, the handwritten German 1 closely resembles a 7 and is 
interpreted as such by handwriting recognition software developed in the U.S. Or 
handwriting recognition engines may lack robustness in interpreting numbers. For 
example, handwriting recognition engines that are trained on a specific user's input may 
20 not correctly interpret hastily written numbers by the specific user and can not at all 
interpret numbers from a different user. In some cases, the number 8 could easily be 
. interpreted as the letter B. 

[006] Some systems have tried to solve these problems with time recognition 

25 by providing graphical user interfaces through which a user may select time information 
from pull-down menus. In these systems, a more complex input/output device than the 
capture device must be used to display the graphical user interfaces. Such a device 
could be expensive and too bulky to carry, particularly for field surveys, field inventory, 
etc., for which the capture device is ideally suited. 

30 

[007] Accordingly, there is a need in the art for a simple and natural way to 

improve the recognition accuracy of time information entered by a user onto printed 
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paper forms attached to capture devices independent of the user who inputs the 
information. 

SUMMARY OF THE INVENTION 
5 [008] Embodiments of the present invention provide a simple and natural 

method to improve recognition of time information. These embodiments provide time 
information by making marks on a paper data form corresponding to the hour, minute, 
and AM/PM, while a capture device captures when and where on the paper data form 
the marks were made. A method includes a processor receiving capture data from the 

10 capture device, where the capture data represents positions of a set of marks made on 
paper overlaying a face of the capture device. The method further includes the 
processor comparing the capture data with one of a plurality of unique positions stored 
in memory in association with a plurality of clock times printed on the paper and 
retrieving from memory the clock time associated with the unique position that matches 

15 the capture data. The method then includes the processor storing the retrieved clock 
time in memory as the set of marks made on the paper. The capture data is 
advantageously generated by simply using a piece of paper and the capture device 
without having to rely on more complex, bulky devices with graphical user interfaces. 

20 [009] x Embodiments of the present invention also provide a system through 
which time information may be recognized. The system may include a memory and a 
processor for receiving capture data corresponding to a set of marks made on a paper 
data form attached to a capture device and mapping the capture data to a time. 

25 BRIEF DESCRIPTION OF DRAWINGS 

[0010] FIG. 1 is an exemplary computer network used to recognize the time 
according to embodiments of the present invention. 

[0011] FIG. 2 is an exemplary computer used to recognize the time according to 
30 embodiments of the present invention. 
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[0012] FIG. 3 is an exemplary paper data form that includes time information to 
be filled out according to an embodiment of the present invention. 

[0013] FIG. 4 is an exemplary data capture format according to an embodiment 
5 of the present invention. 

[0014] FIG. 5 is a flowchart of an embodiment of a method according to the 
present invention. 

10 DETAILED DESCRIPTION 

[0015] Embodiments of the present invention provide a method and system for 
recognizing time information from a paper data form attached to a capture device. 
Time information on the paper may be in the form of a clock face minus the hands and 
a pair of boxes for "AM" and "PM." A user may simply draw the hands on the clock and 

15 fill in one of the boxes corresponding to the correct time. The capture device may 
digitally capture the pen strokes the user makes when drawing the hands and filling in 
the box and upload this capture data to a computer for time recognition according to 
embodiments of the present invention. Exemplary applications of these embodiments 
include field surveys, field inventory, and other applications where paper forms are the 

20 predominant way data is recorded and device portability and ease of use are preferable. 

[0016] In embodiments of the present invention, the computer's processor may 
receive the capture data from a capture device to which a paper data form was 
previously attached. The capture data format may be a time ordered sequence of (x,y) 

25 coordinates, indicating when and where on the data form a set of marks (or pen 

strokes) was made to fill in the correct time. This capture data may then be compared 
to a set of predefined coordinates stored in the computer's memory which indicates the 
expected positions of each hour, minute, and AM/PM on the data form. The predefined 
data that matches the capture data may be determined and the corresponding hour, 

30 minute, and AM/PM stored in memory for later use; hence, the time is recognized. In 
an alternate embodiment, the capture data format may be a sequence of vectors (x, y, 
t) or any format that appropriately represents the user's pen strokes. 
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[0017] Instead of having to rely on handwritten numbers or a variety of time 
formats, embodiments of the present invention may use unique positions of time 
information printed on paper to determine the- time. Accordingly, these embodiments 
5 advantageously provide a simple and natural way to accurately recognize time 
information. Problematic handwriting recognition need not be done, resulting in 
reduced data ambiguity and improved speed of data entry. 

[0018] FIG. 1 shows an embodiment of an exemplary network that may be used 
10 to implement embodiments of the present invention. The exemplary network system 
100 may include, but is not limited to, a computer network 110, computers 120-1 
through 120-C, where C is an integer, capture devices 160-1 through 160-B used by : 
users 170-1 through 170-B, where B is an integer, to input time information, a server 
140, and a database 150 storing various time coordinates used by the computers. 
15 These components may be linked to the network 110 via network links 115. The 

network 110 may be a LAN, WAN, Internet, or any like structure capable of connecting 
components and transmitting data. The network links 115 may include physical wiring, 
wireless connections, or any like' transmission configuration capable of transmitting data. 
Alternatively, a capture device 160 may be directly linked via a wireless link 117, a COM 
20 cable 119, or any like connector, to a computer 120. 

[0019] The capture device used in embodiments of the present invention may 
include a portable input device whose appearance and operation resembles that of a 
traditional clipboard. The capture device may include a flat panel onto which a piece of 

25 paper may be attached and pens used to write on the paper thereby entering data to 
the capture device. The paper generally replaces a graphical user interface that is 
included in most input devices. So, typically, the capture device does not include a 
graphical user interface. The pen strokes made on the paper may be stored in memory 
on the capture device for later uploading to a computer via a modem, cable, or other 

30 transmission device in communication with a port of the capture device. An example of 

v 

the capture device is the CrossPad ™ manufactured by IBM. 
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[0020] In an embodiment, the capture device may include software for 
interacting with a user and for uploading capture data to the computer. The capture 
device may include a series of built-in buttons that may be configured to initiate given 
commands. For example, capture data may be uploaded to the computer via the 
5 wireless link, COM cable, or the like, by the user pressing some of the buttons to initiate 
the upload process. After the upload completes, the user may delete the capture data 
from the capture device. The capture device may include a small text-based display to 
show short text messages to the user. 

10 [0021] In an alternate embodiment, the capture device may include local 
intelligence for performing recognition and uploading the recognized data to the 
computer for further processing. 

[0022] Since digital handwriting capture is not limited to physical flat panel 
15 devices, in another alternative embodiment the capture device may include electronic > 
reusable paper, for example. Electronic reusable paper is designed to have the look and 
feel of normal pgper, except that it contains tiny sensor network technologies that 
provide digital display and capture of handwritten notes. Similar to a flat panel device, 
data can be captured, except that in the case of electronic reusable paper that data is 
20 collected and stored by the paper itself. Data collection from electronic reusable paper 
may be implemented in many ways, including attaching the paper to a clipboard 
containing the electronics required to retrieve data from the electronic reusable paper 
and forwarding the data obtained using standard methods. An example of electronic 
reusable paper is SmartPaper manufactured by Gyricon LLC. 

25 

[0023] FIG, 2 is a block diagram of an exemplary computer that can implement 
embodiments of the present invention. The computer 200 may receive capture data 
from the capture device according to embodiments of the present invention. The 
computer 200 may include, but is not limited to, a processor 220 provided in 
30 communication with a system memory module 230, a storage device 240, and an I/O 
device 250. The processor 220 may perform data recognition with the capture data 
received from the capture device. The memory 230 may store program instructions to 
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be executed by the processor 220 and also may store variable data generated pursuant 
to program execution. In practice, the memory 230 may be a memory system including 
one or more electrical, magnetic, or optical memory devices. The I/O device 250 may 
include a docking station for interface to the capture device 160 to receive the capture 
5 data and transmit any other appropriate data between the capture device 160 and the 
computer 200. 

[0024] In embodiments of the present invention, a paper form may have printed 
thereon time information, including hour, minute, and AM/PM. Each possible value for 
10 hour, minute, and AM/PM may occupy a unique spatial position on the paper form. 
When a user marks a time on the form by selecting an hour, minute, and AM/PM, the 
positions of the user's marks are unique to that time. These positions may be captured 
by the capture device and later uploaded to a computer for processing. Hence, the 
computer may determine the clock time based on these unique positions. 

15 

. • [0025] FIG. 3 is an example of a paper data form in which the time information 
is printed in unique positions as described. In this example, the data form 300 may 
include, but is not limited to, time information comprising a clock face 340 and a pair of 
boxes 350 listing "AM" and "PM." Each component 340, 350 occupies a unique position 
20 on the data form 300, which, when marked by the user, may be used to determine clock 
time based on that position of the mark. FIG. 4 will further illustrate this. 

[0026] The data form 300 may also include date information comprising a first 
group of boxes 310 listing the 12 months in a year, a second group of boxes 320 listing 
25 31 days in a month, and a third group of boxes 330 listing a current span of years. The 
data form 300 may also include a survey 360 to be filled out and the identification 370 
of the data form. 

[0027] The data form 300 may be attached to the capture device 160 and the 
30 correct time filled in by drawing the hands on the clock face 340 and filling in one of the 
boxes 350. The coordinates of the marks made may be recorded on the capture device 
160 and later uploaded to the computer 120 for processing according to embodiments 
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of the present invention. An hour and minute hand may be drawn outward from the 
center point of the clock face 340 to the appropriate positions. As with traditional 
clocks, the minute hand should be longer than the hour hand. In the instance when the 
two hands overlap^ for example, at 1:05, 2:10, etc., the minute and hour hands may be 
5 drawn on top of each other with the minute hand drawn longer. In this case, the 
capture device 160 will capture the two drawn hands with similar, but different, outer 
end point coordinates. The box 350 may be filled in by shading the.entire box or 
making a checkmark, an %" or any discernible indicator that the box has been selected. 

10 [0028] To accurately capture the correct coordinates of the time information, the 
data form 300 should first be properly oriented on the capture device 160 and firmly 
attached to prevent movement. In one embodiment, a border or like markers may be 
printed on the face of the capture device indicating where the data form 300 should be 
attached. Alternatively, the data form 300 may have printed in each corner a hash mark 

15 or like markers. A user first would write on the paper form at the hash marks prior to 
marking the form with time information. The coordinates of these hash marks may be 
captured and uploaded to the computer where used as reference points for the 
coordinates of the time information. 

20 [0029] Different data forms 300 may have the time information 340, 350 in 

different positions and/or formats on the form. Hence, the identification 370 of the data 
form may be uploaded to the computer 120 so that the computer 120 may retrieve the 
appropriate predefined coordinates. In one embodiment, the form identification 370 
may have a uniquely positioned box associated with it to be filled in by the user. The 

25 coordinates of the filled-in identification box may indicate to the computer 120 which 
data form is being used. 

[0030] It is to be understood that the layout of the time information on the form 
is not limited to a clock face and an AM/PM box, as shown in FIG. 3. The time 
30 information may have any layout that allows the user to mark a time on a form without 
having to rely on handwriting recognition software. For example, in an alternate data , 
form, the hour and minute information may include a first group of boxes, one for each 
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of 12 hours on a clock face, and a second group of boxes, one for each of 60 minutes. 
One box from each group and the AM/PM box may be filled in for the correct time. 

[0031] FIG. 4 illustrates an example of the capture data format that may be used 
5 in embodiments of the present invention. , In this example, the user drew the minute 
hand 410 and the hour hand 420 and filled in the AM/PM box using an "X" mark 430 on 
the paper data form to indicate the time 8:01 AM. The capture device digitally captured 
the hands 410, 420 and the "X" mark 430 as time ordered coordinates. Here, (al,bl) 
and (a2,b2) are the end coordinates for the minute hand 410, (a3,b3) and (a4,b4) are 

10 the end coordinates for the hour hand 420, and (a5,b5) are the coordinates for the "AM" 
box. The user entered the time information in the following order: drew the minute 
hand, drew the hour hand, checked the AM box. Hence, the corresponding coordinates 
were uploaded to the computer in that order, as illustrated by 440. The processor 220 
may map the unique coordinates to the hour, minute, and AM/PM, i.e., 8:01 AM, to 

15 which they belong. The processor 220 may further use the ordering as indication of 
when the marks were made, i.e., relative to each other. 

[0032] When drawing lines in embodiments of the present invention, the capture 
device 160 may digitally capture one or more (x,y) coordinates along the trajectory of 
20 the drawn line, depending on the application. In this example, the capture device 160 
captures the two end point coordinates of the minute and hour hands 410, 420. The 
capture device 160 also captures one coordinate pair for the "X" mark 430. In general, 
"X" mark 430 may provide four coordinate pairs, two for each line of the "X." 

25 [0033] FIG. 5 is a flowchart of an embodiment of a method for recognizing time 
information according to the present invention. The processor 220 may receive (505) 
capture data from the capture device 160. As stated previously, the capture data may 
include, but is not limited to, a time-ordered set of coordinates indicating when and 
where on a paper data form a set of marks was made to fill in the correct time. The 

30 processor 220 may then compare (510) the set of coordinates with a set of predefined 
coordinates in memory 230 or storage 240 to find a match for the hour, minute, and 
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AM/PM. The predefined coordinates may define the expected unique positions of marks 
on the paper data form for each hour, minute, and AM/PM in a day. 

[0034] In a system where a variety of paper data forms may be used, the 
5 processor 220 may also receive the form identification from the capture device 160. 
The form identification may be indicated by a set of coordinates, vectors, etc., indicating 
where on the paper data form the identification mark was made. Prior to retrieving the 
predefined coordinates, the processor 220 may compare the form identification 
coordinates with predefined identification coordinates to find a match. The. processor 
10 220 may then determine the predefined coordinates in memory 230 or storage 240 

based on the form identification and compare (510) the captured set of coordinates with 
these determined predefined coordinates. 

[0035] The only requirements for drawing the hour and minute hands 410, 420 
15 are that the hands 410, 420 meet in the center of the clock face and that the minute 
hand 410 be longer than the hour hand 420. Alternatively, the minute hand 410 may be 
wider than the hour hand 420 or any like characteristic that may be used to distinguish 
between the hands. Accordingly, at least two coordinates in the time ordered sequence 
440 may be at or close to the center of the clock face. Coordinates immediately 
20 following each of these center coordinates in the sequence may be the outer end points 
of the minute and hour hands 410, 420. In the example in FIG. 4, the two center 
coordinates (al,bl) and (a3,b3) have immediately following coordinates (a2,b2) and 
(a4,b4), respectively, which indicate the outer end points of the minute and hour hands 
410, 420. Using well-known line calculations, the processor 220 may calculate the 
25 lengths of the drawn hands 410, 420 and determine the longer hand. The processor 
220 may then designate the outer end point corresponding to the longer hand as the 
minute hand 410 and the outer end point corresponding to the shorter hand as the hour 
hand 420. These designated points may be used later for further processing. 

30 [0036] In an alternate embodiment, the minute hand 410 need not be longer 
than the hour hand 420. The hands 410, 420 may be differentiated from each other by 
the order in which they are drawn, where the hour hand 420 is generally drawn first. 

. ~> 
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[0037] In another alternate embodiment, the processor 220 may determine the 
time using right triangle geometry calculations, rather than predefined coordinate 
matching. In this instance, the processor 220 may use the hand lengths or drawing 
5 order to differentiate between the hands 410, 420. The processor 220 may then 

determine the angle between the hands 410, 420. Using the angle and the hands' end 
point coordinates, the processor 220 may then calculate to which numbers on the clock 
face the hands 410, 420 are pointing. Hence, the time may be recognized. 

10 [0038] Embodiments of the present invention provide a way for the user to 
correct the time information when the user incorrectly or erroneously fills in the paper 
form. The user may cross out the incorrect hand or box in question and then draw the 
correct hand or fill in the correct box. In these instances, the capture device 160 records 
more than two sets of coordinates in the clock face 340 and/or coordinates for both 

15 boxes in the AM/PM group 350. For example, if the user drew the hour hand at 5:00 
and then mistakenly drew the minute hand at :05, crossed it out, and correctly drew it 
at :10, there would be four discrete sets of captured coordinates in the clock face— a first 
set made when drawing the hour hand, a second set when drawing the :05 minute 
hand, a third set when crossing out the :05 minute hand, and a fourth set when drawing 

20 the :10 minute hand— where each set would be spaced' out in time. The processor 220 
will have incorrectly received the multiple minute hand coordinates. Using well-known 
line calculations, the processor 220 may determine which lines intersect, i.e., which 
hand has been crossed out, and then eliminate those coordinates. The processor 220 
may then, using well-know line calculations, calculate the lengths of the remaining 

25 drawn hands 410, 420 and determine the longer hand. The processor 220 may 

designate the outer end point corresponding to the longer hand as the minute hand 410 
and the outer end point corresponding to the shorter hand as the hour hand 420. 

[0039] In the case of the mistaken AM/PM, the processor 220, using well-known 
30 line calculations, may determine which of the AM/PM boxes intersects with the cross-out 
line and eliminate the coordinates of that box. The processor 220 may use the 
remaining coordinates later for further processing. 
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[0040] So, according to embodiments of the present invention, if the processor 
220 receives (520) multiple coordinates for the minute hand, the hour hand, and/or 
AM/PM, i.e., finds multiple matches with the predefined coordinates, the processor 220 
5 may determine (525) the correct set of coordinates as previously described or in any 
other manner for determining and eliminating multiple coordinates and, hence, the 
correct predefined coordinates, 

[0041] Next, the processor 220 may retrieve (530) from memory or storage the 
10 time associated with the predefined coordinates that match the captured set of 

coordinates. The processor 220 may then store (540) the hour, minute, and AM/PM as . 
the time the user marked on the form. 

[0042] The processor 220 may alternatively retrieve the predefined coordinates 
15 from memory or storage, one at a time or together, prior to the comparison with the 
captured set of coordinates and then store the hour, minute, and AM/PM that match the 
captured set of coordinates as the time the user marked on the form. 

[0043] In an alternate embodiment, the capture device 160 may perform both 
20 the data capture and the time determination. After which, the capture device 160 may 
upload the time to the computer 120 for further use or storage. 

[0044] Embodiments of the present invention may be implemented using any 
type of computer, such as a general-purpose microprocessor, programmed according to 
25 the teachings of the embodiments. The embodiments of the present invention thus also 
includes a machine readable medium, which may include instructions used to program a 
processor to perform a method according to the embodiments of the present invention. 
This medium may include, but is not limited to, any type of disk including floppy disk, 
optical disk, and CD-ROMs. 

30 

[0045] It may be understood that the structure of the software used to 
implement the embodiments of the invention may take any desired form, such as a 
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single or multiple programs. It may be further understood that the method of an 
embodiment of the present invention may be implemented by software, hardware, or a 
combination thereof. 

[0046] The above is a detailed discussion of the preferred embodiments of the 
invention. The full scope of the invention to which applicants are entitled is defined by 
the claims hereinafter. It is intended that the scope of the claims may cover other 
embodiments than those described above and their equivalents. 
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